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Abstract — We investigate the privacy amplification problem in 
which Eve can observe the uniform binary source through a 
binary erasure channel (BEC) or a binary symmetric channel 
(BSC). For this problem, we derive the so-called expurgation 
exponent of the information leaked to Eve. The exponent is de- 
rived by relating the leaked information to the error probability 
of the linear code that is generated by the linear hash function 
used in the privacy amplification, which is also interesting in its 
own right. The derived exponent is larger than state-of-the-art 
exponent recently derived by Hayashi at low rate. 

I. Introduction 

In information theoretic key agreement problem JT), J2), 0, 
[4 1, [5 1, [6 1, legitimate parties need to distill a secret key from 
a random variable in the situation such that an eavesdropper 
can access to a random variable that is correlated to the 
legitimate parties' random variable. The privacy amplification 
is a technique to distill a secret key under the situation by using 
a (possibly random) function |7|. The security of distilled key 
is evaluated by various kinds of measures. In this paper, we 
focus on the leaked information, which is the mutual infor- 
mation between the distilled key and eavesdropper's random 
variable (the so-called strong security J8), because it is 
the strongest notion among security criterion [4] (see also [10 
Appendix 3]). 

The privacy amplification is usually conducted by using a 
family of universal 2 hash functions ifTTl . In Q, Bennett et. al. 
evaluated ensemble averages of the leaked information for 
universal 2 families, and derived an upper bound on the leaked 
information by using the Renyi entropy of order 2. In [12|, 
Renner and Wolf evaluated ensemble averages of the leaked 
information for universal 2 families, and derived an upper 
bound on the leaked information by using the smooth min- 
imum entropy. In iflOl . Hayashi evaluated ensemble averages 
of the leaked information for universal 2 families, and derived 
a parametric upper bound on the leaked information by using 
the Renyi entropy of order 1 + 6. Concerning the exponential 
decreasing rate of the leaked information, the exponent derived 
by Hayashi's bound is state-of-the-art. 

In noisy channel coding problem, the exponential decreasing 
rate of the error probability is also regarded as an important 
performance criterion of codes, and has been studied for a 
long time. The best exponent at high rates is the one derived 
by Gallager's random coding bound |fl3l . However, Gallager's 
exponent is not tight in general, and can be improved at low 
rates because the random code ensemble involves some bad 



codes and those bad codes become dominant at low rates. The 
improved exponent by expurgating those bad codes is usually 
called the expurgation exponent [13|, [14|. Similar improved 
exponents are also known in the context of the Slepian-Wolf 
coding lfl5l . lfl6ll or the quantum error correction IfTTl . 

The purpose of this paper is to show a security analog 
of above results, i.e., to derive an improved exponent of the 
leaked information in the privacy amplification at low rates. 
For this purpose, we concentrate our attention on the case such 
that the random variable possessed by the legitimate parties is 
the binary uniform source and the function used in the privacy 
amplification is a linear matrix. 

We first consider the case such that the eavesdropper's 
random variable is generated via a binary erasure channel 
(BEC). For this case, we first relate the leaked information 
to the maximum likelihood (ML) decoding error probability 
of the linear code whose generator matrix is the one used in the 
privacy amplification. Then an improved exponent is derived 
by using the result of the expurgation exponent of linear codes. 

It should be noted that a similar approach to relate the 
leaked information to the erasure error correction has been 
appeared in iTTsTl - However in this paper, we directly relate 
the leaked information to the ML decoding error probability, 
which enables us to derive the improved exponent. It should 
be also noted that the approach in this paper is completely 
different from the error correction approach conventionally 
used to prove the so-called weak security and the problem 
pointed out in |19| does not apply to our approach. 

Next, we consider the case such that the eavesdropper's 
random variable is generated via a binary symmetric channel 
(BSC). For this case, the technique used in the BEC case 
cannot be directly applied. Thus, we first reduce the BSC 
case to the BEC case by using the partial order between 
BSCs and BECs. The reduction turns out to be quite tight. 
Indeed, the exponent derived via this reduction is as good as 
Hayashi's exponent below the critical rate, and strictly better 
than Hayashi's exponent below the expurgation rate, which 
resemble the relation between the expurgation exponent and 
the random coding exponent of the noisy channel coding. Our 
results suggest that the privacy amplification with a universal 
2 family is not necessarily optimal. 

The rest of the paper is organized as follows. We first 
explain the problem formulation of the privacy amplification in 
Section|II] Then, we consider the BEC case and the BSC case 
in Sections ITII1 and HVl respectively. Conclusions are discussed 



in Section [Vl 

II. Problem Formulation 

Let (X n , Z n ) be a correlated i.i.d. source with distribution 
Pxz- The alphabet is denoted by X x Z. In the privacy ampli- 
fication problem, we are interested in generating the uniform 
random number on S n by using a function /„ : X n — > S n . 
The joint distribution of the generated random number and the 
side-information is given by 

Ps n zn(s n ,z n )= £ p xz(x n ,z n ), 

where f- x (s n ) = {x n e X" : f n (x n ) = s n }. 

The security is evaluated by the leaked information 

I(f n ) = I(S n ;Z n ) 

where /(•; •) is the mutual information and we take the base 
of the logarithm to be e. 

For given rate R > 0, we are interested in the exponential 
decreasing rate of I(f n ), i-e., 

E(R;X\Z) 

= sup | lim inf — — log I(f n ) '■ lim inf — log |<S n | > R \ . 

I n— >oo 71 n— >oo n j 

In the privacy amplification problem, we typically use the 
universal 2 hash family. 

Definition 1: A family T n of functions /„ : X n — > S n is 
called universal 2 if 

Pt{F n (x n ) = F n (x n )} < JL 

for every x n ^ x n , where F n is the uniform random variable 
on T n - 

For parameter 8, let 

*Ij(9;X\Z) = -log ^ Pzx(x,z) 1+e P z (z)- e 

= -logY Pxz(x,z)exp [0 log P x \z(x\z)] . 

Hayashi derived the following lower bound on E(R; X\Z). 

Proposition 2 ([10]): For any universal 2 hash family F n , 
we have 

E(R;X\Z) > lim inf log [/(/„)] 

> E r {R-X\Z) 

:= max \ip(6;X\Z)-6R), 

0<9<l 

where Ejr n means the average over randomly chosen function 
from T n . 




Fig. 1. The channel considered in Fig. 2. The virtual channel con- 
Section HH] sidered in Section ITlTl 

III. Side-Information via Binary Erasure Channel 

In this section, we assume that X is the uniform binary 
source and Z is the output of the binary erasure channel 
(BEC) with erasure probability e, i.e., Pxz(x,x) = ^rp and 
Pxz(x>t) = §> where ? represent the erasure symbol (see 
Fig. [TJi. For given sequence z n , let J{z n ) C {1, . . . , n} be 
the set of those indices such that Zj =?. When the sequence 
z n is obvious from the context, we abbreviate J(z n ) as J . 

In the rest of this paper, we concentrate on the linear 
function f n : X n S n . Thus, we implicitly assume that 
X = F 2 and S n = F§ for some k, where F 2 is the field 
of order 2. Let M n be k x n matrix with entries in F 2 . We 
consider function /„ : x n — > x n M^ and the security criterion 
is denoted by I(M n ). The sequence x% is a subsequence of 
x n that consist of the indices in J, and the matrix Mj is a 
sub-matrix of M n that consist of the columns in J. 

The following lemma was presented by Ozarow and Wyner. 

Lemma 3 ([20]): We have 

H(S n \Z n = z n ) >xwk(M J{zn) ) 
for every z n . 

We consider the virtual BEC with erasure probability 1 — e 
(see Fig. 0, i.e., P Y \x(x\x) — e and P Y \x{t\x) = 1 — e. 
From Lemma [3] we have the following. 

Theorem 4: Let C n be the linear code whose generator 
matrix is M n , and let Pml(C„,1 — e) be the maximum 
likelihood decoding error probabilitjQ of the code C„ over the 
BEC(1 — e). Then, we have 

I(M n ) <nP ML {C n ,l-e). 

Proof: Let m k 6 F 2 is a message to be sent, and the 
encoded message m M n is sent over the BEC(1 — e). Suppose 
that the received signal is y n . If rank(Mj-(j,n)c) = k, then the 
ML decoder output m k , where J{y n ) c = {1, . .. ,n}\J(y n ) 
is the non erased bits. On the other hand, if rank(Mj-( J/ ».)c) < 
k, there are plural messages that are compatible with y n , and 
thus the ML decoder fail to output m k . Therefore, the ML 
decoding error probability can be written as 

P ML (C n ,l-e) 

53 (1 - e) n -\ j0 \s\ jC \l[rmk(Mjc) < k}. 

J rc C{l,...,n} 
'Ties ai'e counted as errors. 



On the other hand , by using Lemma [3] and by noting that 
H(S n ) < n, we have 

J(M n ) < n i 1 - e) n ~ m e m l\pak.(.Mj) < k}. 

JC{l,..,n} 

Thus, we have the assertion of the theorem. ■ 
By using a linear code achieving the Gilbert- Varshamov 

bound, we have the following. 

Corollary 5: There exists a linear function /„ : x n — > 

x™Mj such that 



E(R;X\Z) 



1 



> liminf--log/(/„) 

n— too n 

> lim inf - - log P ML (C„ , 1 - e) 

n— >OQ Jl 



> E x (R,l-e) 



max 
9>i 



0{log2-i?-log(l + (l-e) 1/e )} 



(1) 

(2) 
(3) 
(4) 



Proof: First note that the error probability of the chan- 
nel coding and that of Slepian-Wolf coding (with full side- 
information) are the same for linear code and BEC. Thus, 
Csiszar's linear Slepian-Wolf code result lfl6l implies that 
there exists a code satisfying 

lim inf - - log P ML (C n ,1-e) 

n— too n 



> min 

H(W)>log2-R 



(log2-i?) -H(W) 



E 



- log V p xy(x, y)P X Y {x + W, y) 



■ivy 



= min [-p log(l - e) + (log 2 - R) - h(p)} , 

/i(p)>log 2 — R 

(5) 

where we set Pw(l) = P- Since the objective function of 
Eq. (0 is convex, by introducing 

L(X) := min [-plog(l - e) + (1 + A) (log 2 - R - h(p))\ 
v 

for A > 0, Eq. (0 can be written [|2D as 
maxL(A). 

A>0 

By changing the variable as 9 = 1 + A, Eq. (0 can be also 
written as 

maxL(0 - 1) = max \d{log2- R - log(l + (1 - e) 1/9 )} 

■ 

Note that E X (R,1 — e) is the expurgation exponent for 
BEC(1 - e) 11221 . 

Remark 6: It should be noted that 



E r (R;X\Z) 

= E r (R,l-e) 



max 

o<e<i 



log { (1 - e) + ¥ e} - 9R 



(6) 
(7) 
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Fig. 3. Comparison of E r (R, 1 — e) (dashed line) and E X (R, 1 — e) (solid 
line) for e = 0.5. 
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Fig. 5. The virtual channel con- 
Sectkm Wt consldered m sidered in Section||V] This channel 

' — ' is less noisy than the BSC in Fig.|4] 



Since E r (R, 1— e) is the random coding exponent for BEC(1 — 
£) ED . Hayashi's exponent can be also derived from Theorem 

El 

From Eq. <£3j and Eq. (0 and known facts on the exponents, 
we find that the exponent of PA in Corollary is larger than 
that in Proposition |2]for low R. These exponents are compared 
in Fig. for e = 0.5. We find that E X (R, 1 - e) is strictly 
larger than E r (R, 1 — e) at low rates. 

IV. Side-Information via Binary Symmetric 
Channel 

In this section, we assume that X is the uniform binary 
source and Z is the output of the binary symmetric channel 
(BSC) with crossover probability e, i.e., Pxz(x,x) = 
and Pxz(x,x + 1) = | (see Fig. @J. Let Z be the output 
of BEC(4e(l - e)) with input X. Since BEC(4e(l - e)) (see 
Fig. is less noisy than BSC(e) ||23l , we have 

I(S n ;Z n ) <I(S n ;Z n ). 

Thus, Corollary0can be applied to the case considered in this 
section. 

Theorem 7: Let Z be the output of BEC(4e(l - e)) with 
input X. Then, we have 

E{R;X\Z) > E[R;X\Z) 

> E x {R,l-Ae{l-e)). 
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Fig. 6. Comparison of E r (R, X\Z) (dashed line) and E X (R, 1 -4e(l -e)) 
(solid line) for BSC(O.ll). 




Fig. 7. Comparison of £ r (K, X\Z) (dashed line) and E X (R, 1 — 4e(l— e)) 
(solid line) for BSC(0.25). 



Hayashi's exponent for BSC(e) is 

EJR;X\Z)= max f- log { (1 - e) 1+e + e 1+e \ - 9R] . 

o<e<i J J 

The exponents are compared in Fig. [6] and Fig. [7] for e = 0.11 
and 0.25 respectively. 

Let i? cr (e) be the critical rate, i.e., the largest rate such that 
the optimization in E r (R; X\Z) is achieved by 9 = 1. Then, 
for R < R cr (e), we have 

E r {R;X\Z) = - log{(l - e) 2 + £ 2 } - i?. 

On the other hand, let R x (e) be the expurgation rate, i.e., the 
smallest rate such that the optimization in E X (R, 1— 4e(l— e)) 
is achieved by 9 = 1. Then, for R x (e) < R, we have 

E X {R,1 - Ae{l - e)) 

= log2-i?-log(l + l-4e(l- £ )) 
= -log{(l- £ ) 2 +e 2 )}-i?. 

Thus, for R x (e) < R < R cr (e), E r (R;X\Z) = E X (R,1 - 
4e(l — e)), which can be also observed in Fig. [6] and Fig. [7] 



We also find that E X (R, 1 — 4e(l — e)) is strictly larger than 
E r (R\X\Z) at low rates. 

V. Conclusion 

For the BEC case and the BSC case, we derived the 
expurgation exponent of the leaked information in the privacy 
amplification. The technique to relate the leaked information to 
the ML decoding error probability heavily relies on the specific 
structure of the BEC. Thus, to derive the expurgation exponent 
for general cases, a method to expurgate bad functions directly 
might be needed. 

Hayashi derived a quantum counter part of Proposition [2] in 
ll24l . It is also interesting to derive the expurgation exponent 
in the privacy amplification for quantum adversary. For the 
case such that the eavesdropper's information is generated via 
the complementary channel of a Pauli channel, the technique 
to relate the leaked information to the ML decoding error 
probability is already known Il25lr4. and it is not difficult 
to derive the expurgation exponent. In general, more refined 
technique is needed. These topics will be investigated in 
elsewhere. 
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